A6-A182  MS  OPTIMAL  CONTROL  WITH  DIMINISHING  ANO  ZERO  COST  FOR 
CONTROL (U)  CARNEGIE-MELLON  UNIV  PITTSBURGH  PA 
S  E  SHREVE  ET  AL  20  SEP  85  AFOSR-TR-87-0950 
UNCLASSIFIED  AFOSfi-82-0259  F2G  1222 


E  NO 

V.rJ 

DTK. 

m  FILE  copy  . 


SBdWR'TY  CtA$SiFi CATION  OF  THIS  FAGS 


REPORT  DOCUMENTATION  PAGE 

I  lb.  RESTRICTIVE  MARKINGS 


AD-A182  805 


4.  PCMFOAMING  ORGANIZATION  REPORT  NUMBER(S) 


3.  OiSTRIBUTlON/AVAiLABlLlTV  OF  REPORT 

Approv-  i  f.  r  puf-lic  .t,;eft,a; 

dictritutigaunliaiitt-a 


6ft.  N AMS  OF  PERFORMING  ORGANIZATION 


B.  OFFICE  SYMBOL 
Ilf  applicable) 


niversit 


6c.  AOORESS  (City.  State  on*  ZIP  Code) 


500  Forbes  Ave.  Or" 

Pittsburgh,  Pennsylvania  15213-3890  0  ^ 

- - -  -  -  —  j 


•ft.  NAME  OF  FUNOING/SPONSORING  SB.  OFFICE  SYMBOL  9.  PROCUREMENT  INSTRUMENT  lOENTIPlCAlSW^NUMBE  R''-J 

ORGANIZATION  (If  applicable)  • 

AFOSR  KM  AFOSR-82-0259 


~b.  AOORESS  (City,  State  and  ZIP  Code) 


AFOSR 


Sc.  AOORESS  (City.  State  and  ZIP  Code) 

titosH/ta 

B1c?y4>o  element  nc 

Eoi  1 <  •»  i -■>  ~  -i  61102F 

11.  TITLE  I Include  Security  Claeaificationf  ' J 

Optimal  Control  with  Diminishing  &  Zero  Cost  for  Control 

12.  personal  authorisi  — — 

S.E.  Shreve  &  V.J.  Mizel 


13a  type  of  report  .  ...... 

Final  from7/1/82  r0  9/30/8 j  ~  7/1/8 


^s.  supplementary  notation 


10.  SOURCE  OF  FUNDING  NOS. 


PROGRAM  PROJECT 

element  no.  no. 

61102F  2304 


PROJECT 

TASK 

NO. 

NO. 

2304 

Al 

WORK  UNIT 
NO. 


14  DATE  OF  REPORT  (Yr .  Mo..  Day) 

7/1/82 


15.  PAGE  COUNT 


COSATI  COOES 


18  SUBJECT  TERMS  I Continue  on  reuerte  if  neceuary  and  identify  by  block  number) 

f  mo'  v 5  L  y 


ABSTRACT  /Continue  on  reverte  if  necessary  and  identify  by  block  number) 

. —  .-  Research  under  this  grant  focused  on  singular  and  bang-bang  stochastic  control  and 

singular  deterministic  control.  The  investigatiors  established  the  connections  between 
a  general  Monotone  Follower  problem  and  a  stopping  probleii:  the  spatial  derivative  of  the 
value  function  of  the  Monotone  Follower  problem  is  the  opitmadt  risk  function  from  the 
stopping  problem.  Similar  results  were  obtained  for  the  general  Reflected  Follower 
problem.  This  connection  between  singular  control  and  optimal  stopping  has  proved  useful 
in  establishing  optimal  stopping  times  via  the  corresponding  control  problem  and  permits 
application  of  analytical  and  numerical  methods  from  optimal  stopping  to  singular  control 
problems.  Nine  papers  were  published  under  this  grant,  including^Connections  between 
optimal  stopping  and  singular  stochastic  control  I”',  ■ “Trivariate  density  of  Brownian 
motion,  its  local  and  occupation  time,  with  application  to  stochastic  control*1'?"  and 
-“A  stochastic  control  problem  with  different  value  functions  for  singular  and  absolutely 
continuous  control.' _ _ 


20.  oistribution/availabiuty  of  abstract 
UNCLASSIFIEO/UNLIMITEO  □  SAME  AB  RPT  UOT.c  USERS  □ 


22a  NAME  OF  RESPONSIBLE  INDIVIDUAL 
Ma j .  James  Crowley 


DO  FORM  1473,  83  APR 


21  ABSTRACT  SECURITY  CLASSIFICATION 


22b  TELEPHONE  NUMBER 
i Include  Area  Code) 

(202)767-5025 


22c  OFFICE  SYMBOL 

Nil 


EDITION  OF  1  jAN  73  IS  OBSOLETE. 


SECURITY  CLASBlf  I^A^ION  of  this  PAGE 


afosr 


TR.  8  7  -  0  9  50 


Final  Scientific  Report  on 
AFOSR  Grant  Number  8-1—Q4 5ft  '  j  .1  -  I 
July  1.  1982  -  Sept.  30.  1985 


Optimal  Control  with  Diminishing  and  Zero 
Cost  for  Control 

Principal  Investigators:  S.E.  Shreve  and  V.J.  Mizel 


Accession  For 

ntis  gra&i 

dtic  TAB 
Unannounced 
Justif  loatioi 

* 

□ 

* 

Hr*  _ -  ■  — 

nietributlon/ 

Aval] 

Dlst 

m 

Lablllt 

Avail 

Spec 

y  Codes 
rnd/or 

Lai 

BBS 


JUL3  0 1987 


1 


Final  Scientific  Report  on 
AFOSR  Grant  Number  81-0159 
July  1.  1982  -  Sept.  30.  1985 

Optimal  Control  with  Diminishing  and  Zero 
Cost  for  Control 

Principal  Investigators:  S.E.  Shreve  and  V.J.  Mizel 

Research  under  this  grant  focussed  on  singular  and  bang-bang 
stochastic  control  and  singular  deterministic  control.  In  each 
of  the  following  sections,  we  briefly  outline  these  p'roblems  and 
describe  progress  made  on  them.  The  resulting  publications  are 
attached.  The  final  section  describes  a  book  whose  authorship 
was  partially  supported  by  this  grant. 

Section  1 ■  Singular  Stochastic  Control 

The  first  singular  control  models  were  those  of  spaceship 
control  treated  by  Chernoff  [10]  and  Bather  and  Chernoff  [5,6]. 
Bather  and  Chernoff  [5]  cast  their  spaceship  control  problem  in 
the  following  form: 

rT 


Minimize : 

1—1 

w 

1  ^  1  L.  2, 

f=7  d?s  +  2  k  XT] 

0 

Subject  to: 

xt  = 

*  X  +  Wt  -  ft  + 

0  <  t  <  T 

In  this  formulation,  {w^;  0  <  t  <  T}  is  a  standard  Brownian 
motion,  {?t:  0  <  t  <  T}  is  a  nondecreasing  control  process 
adapted  to  the  filtration  of  the  Brownian  motion  and  satisfying 


£q  =  0,  and  {ft:  0  <  t  <  T}  is  a  nondecreasing  adapted  process 
which  causes  {x^:  0  ^  t  £  T}  to  reflect  at  the  origin,  i.e., 

C  =  -  min[(x-f  +  w  )  A  0  ]  ;  0<t<T. 

0< s <  t  s  s 

Such  a  singular  control  problem  in  which  the  state  process  is 
reflected  at  the  origin  will  be  called  a  Reflected  Follower 
Problem.  Bather  and  Chernoff  conjectured  that  the  optimal  f 

c 

should  be  the  minimal  process  which  keeps  the  state  trajectory 
below  a  time-varying  boundary  f(t),  i.e., 

f  =  max  [(x  +  w  +  f  -  f ( t ) )  V  0];  0  <  t  <  T. 

0<  s  <  t  t 

One  then  would  expect  the  value  function  for  this  problem  to 
satisfy  the  Hami 1  ton- Jacobi-Be 1 lman  equation 

V t ( t . x )  +  |  vxx(t,x)  =0,  0  <  x  <  f(t).  0  <  t  <  T, 

/ 

with  boundary  conditions 

Reflection  at  origin:  V  (t,0)  =0;  0  <  t  <  T 

Terminal  cost:  V(T,x)  =  ^  kx^;  x  2  0. 

On  the  boundary  {(t,x);  x  =  f(t),  0  <  t  <  T},  the  marginal  cost 

of  pushing  should  equal  the  marginal  decrease  in  value,  i.e.. 
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Boundary:  Vx(t.f(t))  =  j~:  0  <  t  £  T. 

Setting  R(t.x)  =  Vx(t,x),  Bather  and  Chernoff  then  derived  the 
condi t ions 


Rt<t-*>  -  k  -  0;  0  *  *  <  0  !  !  !  '■ 

R  (  t  ,  0 )  =  0  ;  0  £  t  <  T . 

R (T . x )  =  kx ;  x  £  0 . 

R(t.x)  =  0  ^  t  <  T. 


and  recognized  these  as  the  free  boundary  conditions  associated 
with  the  problem  of  stopping  a  Brownian  motion  so  as  to  minimize 
the  expected  sum  of  (a)  zero  if  the  motion  reaches  the  origin 
before  terminal  time  T  and  before  it  is  stopped,  (b)  kx  if  the 
motion  does  not  reach  the  origin  and  is  not  stopped  before 
terminal  time  T,  and  arrives  then  at  position  x,  (c)  if 

the  motion  is  stopped  at  time  t  <  T.  They  then  used  this 
optimal  stopping  problem  to  provide  bounds  on  the  free  boundary 
f  . 

In  their  seminal  19S0  paper  [9],  Benes,  Shepp  and 
Witsenhausen  posed  two  singular  control  problems.  One  of  them 
was  the  following  Monotone  Follower  Problem: 


Minimize: 


(x  +  wt  "  ft}' 


where  f  and  w  are  as  in  the  previous  problem.  They  found  a 


4 


boundary  of  the  form  f ( t )  =  5vT- t  such  that  the  optimal  f  is 
the  minimal  process  which  keeps  the  state  trajectory  x  +  w  -  f 
below  f(t).  i.e., 

f  =  max  [(x  +  w  -  f  ( s ) )  V  0];  0  <  t  <  T. 

0<s<  t  s 

They  also  noted  that  this  boundary  is  of  the  same  form  as  that 

found  by  Miroshnichenko  [21]  to  demarcate  the  stopping  and 

* 

continuation  regions  for  the  problem 

*T 

Minimi2e:  E  (x  +  w  )dt, 

where  the  minimum  is  over  stopping  times  t  taking  values  in 
[0.T].  Benes,  et  al..  also  pointed  out  the  connection  with  local 
time;  namely,  the  optimal  f  is.  except  for  a  possible  initial 
jump,  equal  to  the  local  time  of  the  state  process  on  the 
boundary  {(t,x);  x  =  f ( t )  ,  0  £  t  £  T}: 


1  i  m 
eiO 


1 

2t 


t 


1  r  ,  ,  \  r  f  ,  T  (  X  ^  W 

[f(s)-£.f(s)]V  S 

0 


)ds. 


Motivated  by  these  broad  hints  that  some  connection  existed 
between  the  optimal  stopping  and  singular  control,  we  used  this 
grant  to  support  our  investigation  of  this  issue.  In  Karatzas 
and  Shreve  [IS],  we  established  by  probabilistic  means  the 


following  results.  For  the  general  Monoton  Follower  Problem 


Minimize : 


Sube j  c  t  to : 


EC 


h( t . x  ) d t  + 


f(t)dft  +  g(xT)] 


[O.T] 


xt  =  x  +  w  -  0  <  t  <  T, 


where  f  is  nondecreasing,  the  spatial  derivative  of  the  value 
function  is  the  optimal  risk  function  for  the  stopping  problem 


Minimize:  E[ 


hx( 1 .x+wt)dt  +  f(o)l^T<Tj  +  g‘ (x+wt)l{r=Tj] 


where  the  minimum  is  over  all  stopping  times  r  lying  in  [0,T]. 

This  result  was  proved  under  the  assumptions  that  f • 8X  and  hx 

are  continuous,  g  and  h  are  convex  in  x  and  grow  at  most 

polynomial ly .  and  there  exists  an  optimal  process  for  the  control 

problem.  Furthermore,  the  region  of  the  state  space  in  which  no 

control  is  exercised  in  the  control  problem  (the  optimal  f  is 

constant)  is  the  optimal  continuation  region  in  the  stopping 

problem.  In  particular,  an  optimal  stopping  time  exists  in  the 

stopping  problem.  It  was  also  shown  that  if  f(t)  is  positive 

and  bounded  away  from  zero,  h  and  g  are  nonnegative,  and 

sup  g'(x)  £  f(T),  then  an  optimal  process  exists  for  the  control 
x€(R 

problem. 

In  Karatzas  and  Shreve  [20]  we  also  established  similar 
results  for  a  general  Reflected  Follower  Problem.  The  principle 


new  feature  is  that  for  these  problems,  the  Brownian  motion  in 
the  associated  stopping  problem  is  absorbed  at  the  origin,  a  fact 
already  observed  by  Bather  and  Chernoff  [5]. 

One  benefit  of  the  established  connection  between  singular 
control  and  optimal  stopping  is  that  bounds  on  the  continuation 
region  of  the  stopping  problem,  obtained  by  posing  and  solving 
more  and  less  favorable  problems,  translate  into  bounds  on  the 
region  of  inaction  in  the  control  problem.  This  comparison  idea 
goes  back  to  Bather  [4]  and  has  been  used  by  Bather  and'Chernoff 
[5],  Chernoff  [10],  and  Karatzas  [17].  A  second  benefit  of  the 
connection  between  these  problems  is  that,  because  control 
processes  are  more  easily  topologized  than  stopping  times  and  are 
thus  more  amenable  to  the  continuity  and  compactness  arguments 
frequently  used  in  existence  proofs,  it  is  feasible  to  prove  the 
existence  of  optimal  stopping  times  via  the  corresponding  control 
problem.  Such  a  line  of  argument  was  used  in  [18],  and  it  avoids 
the  heavy  regularity  assumptions  usually  required  to  prove  the 
existence  of  optimal  stopping  times.  See,  for  example,  Friedman 
[12].  A  third  benefit  flows  in  the  other  direction:  from 

stopping  to  control.  Because  optimal  stopping  is  a  mature 
research  area,  as  compared  to  singular  control,  there  are  a  large 
number  of  ana lytical  and  numerical  methods  which  can  be  brought 
via  optimal  stopping  to  bear  on  singular  control.  In  particular, 
the  theory  of  numerical  solution  of  Stefan  problems  becomes 


ava i 1 ab 1 e . 


Section  2 .  Bang-Bang  Stochastic  Control 

One  of  the  first  uses  of  Girsanov's  formula  to  solve  a 
stochastic  control  problem  is  due  to  Benes  [7],  This  so-called 
"predicted  miss"  problem  is  to  minimize  the  distance  of  the  final 
state  x(l)  €  from  a  hyperplane  subject  to  the  system 

equa t i on 

»t  pt  pt 

=  x  +  A(s)xsds  +  B( s  )u{ s  ,  xg )ds  +  C(s)dws> 

Jq  Jo  J  o 

where  A,  B  and  C  are  deterministic  matrix  functions  of  time, 
w  is  a  multi-dimensional  Wiener  process,  and  the  control  is 

d  t* 

constrained  by  u:  [0,1]  x  R  -♦  [-1,1]  .  Davis  and  Clark  [11] 
subsequently  showed  how  to  reduce  this  mu  1 t i — d imens i onal  problem 
to  the  one-dimensional  problem: 

2 

Minimize :  Ex  ( 1 ) 

Subject  to 

pt 

xt  =  x  +  u(s,xs)ds  +  w(t) . 

Jo 

|u(t.xt)  I  <  1, 

where  w  is  a  one-dimensional  Wiener  process.  This 
one-dimensional  problem  has  been  solved  by  different  methods  by 
Benes  [8],  Ikeda  and  Watanabe  [16],  Davis  and  Clark  [11], 
Haussman  [13],  Balakrishnan  [1]  and  Shreve  [24].  The  obvious 
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control  law  u(t.x  )  =  -sgnfx^)  is  indeed  the  optimal  one 
Shreve  [24]  computed  the  transition  probabilities  for  the 
controlled  process 


(2.1) 


xt  =  x  - 


sgn(xg)ds  + 


using  Girsanov’s  formula  and  local  time  in  a  way  very  similar  to 
the  method  we  are  about  to  describe  for  the  asymmetric  equation 


(2.2). 


Benes,  Shepp  and  Witsenhausen  [9]  solved  the  related 


problem: 


Minimize'-  E  e  atx^dt. 


Subject  to: 


=  x  +  u(s,x)ds+w 
t  v  s '  t 


-®  <  0Q  <  u(s,xg)  <  0^  <  00  . 


They  proved  the  existence  of  a  number  5  such  that  an  optimal 
control  law  is  given  by 


u(t.xt)  = 


60  if  xt  <  6> 


0:  if  xt  >  5. 


The  resulting  controlled  process  is  governed  by  the  equation 


(2.2) 


x  =  x  +  0(x)ds+w 
t  v  s  •  t 

J0 


where 


0(x)  = 


if  x  <  6 , 


ex  if  X  ^  5. 


This  is  an  asymmetric  version  of  (2.1). 

In  the  work  supported  by  this  grant,  which  appears  in 
Karatzas  and  Shreve  [19],  we  computed  the  transition  density  for 
xt  governed  by  (2.2)  as  follows.  According  to  Girsanov’s 
formula,  the  desired  density  is  given  by 


(2.3) 


P*[xt  €  dy]  .  EX[C(t)l(„t€dy)]. 


where 


at)  = 


exp[  6(wu)dwu 


\  e2(wu)du] 


Our  computation  thus  requires  knowledge  of  the  joint  distribution 


of  ( 


'  t  p  t 

w(t),  0(w  )dw  ,  02(w 


)  du )  . 


Assuming  for  notational  simplicity  •  r.a  • 


(2.4) 


pt 

e2{w  ) d u  =  e .  n  t )  ♦  er  t  ■.  -  r 

U  i 

Jo 


where 


r(t) 


1  ,  , d  s 

{*s>0} 


is  the  occupation  time  for  the  right  half-iine.  A  v 
Tanaka’s  formula  gives 


(2.5) 


*  t 

0(wu)dwu  =  4>(wt)  -  $(w0)  +  (Bj  -B0)L(t) 

Jo 


where 


4>(z) 


'Z 

6(y)dy 

Jo 


and  L(t)  is  the  local  time  of  w  at  the  origin: 


L  (  t  )  = 


i  •  1 

urn  - — 

t  lo  46 


r  t 


0 


{-£  <  W  <  £} 


d  s  . 


a  r  l  a  t  i  o 


The  local  time  appears  because  of  the  discontinuity  of  0 


=  0.  Thus,  our  problem  has  been  reduced  to  a  computation  of  the 
trivariate  density  PX[wt  €  da,  L(t)  €  db ,  T(t)  €  dT]  of 
Brownian  motion,  its  local  time  at  the  origin,  and  its  occupation 
time  of  the  right  half-line.  Karatzas  and  Shreve  [19]  contains  a 
derivation  via  Laplace  transforms  and  the  Feynman-Kac  formula  for 
elastic  Brownian  motion,  and  this  paper  also  sketches  a  proof 
based  on  a  formula  of  D.  Williams  [25]  involving  inverse  local 
times. 


Section  3 .  Singular  Deterministic  Control 

This  grant  supported  an  investigation  into  the  foundations 
of  the  calculus  of  variations  which  led  to  the  discovery  that 
even  for  regu lar  one-dimensional  control  problems  of  Lagrange 
type  the  optimal  solutions  (  "  m  i  n  i  m  i  ze  r  s  ’’ )  can  exhibit  several 
remarkable  features.  For  one  thing,  when  the  asymptotic  behavior 
of  running  cost  of  control  is  markedly  different  at  certain 
status  from  its  behavior  at  others,  then  the  optimal  solution  may 
have  infinite  slope  at  certain  times  and  can  fail  to  satisfy  the 
integrated  form  of  the  Pontryagin  minimum  principle. 

Particularly  striking  is  the  Lavrentiev  phenomenon:  under  the 
above  circumstances,  the  optimal  solution  may  be  such  that  the 
standard  methods  for  approximating  optimal  solutions  of  control 
problems  (penalty  methods,  finite  element  methods  and  the  like) 
will  all  converge  not  to  the  minimizer  itself,  but  instead  to  a 
higher  cost  "pseudominimi zer” .  It  follows  that  these  methods 
cannot  in  general  detect  singular  minimizers.  and  will  instead 


lead  to  overly  pessimistic  cost  estimates  and  suboptimal  control 


policies  (both  of  which  are  not  subject  to  improvement  with 
increasing  computational  accuracy).  This  work  is  reported  in 
Ball  and  Mizel  [2,3]. 


The  generalization  of  the  model  studied  to  allow  for  a  noise 
term  leads  to  a  singular  stochastic  control  problem  of  the  type 
discussed  in  Section  1.  The  Lavrentiev  phenomenon  takes  the  form 
of  a  gap  between  the  best  performance  with  a  non-singular  control 
policy  and  the  best  performance  with  a  singular  control  policy. 
This  work  was  the  basis  for  Heinricher  [14]  and  is  succinctly 
reported  in  Heinricher  and  Mizel  [15]. 

This  grant  also  supported  Mizel  and  Trutzer  [22,23]  for  work 
on  stochastic  hereditary  equations. 

Section  4 .  Martineales,  Brownian  Motion  and  Stochastic 
Differential  Equations 

During  the  work  described  in  this  proposal,  we  have  found 
the  difficult  and  esoteric  concept  of  Brownian  local  time  to  be 
an  i nd i s pen s i b 1 e  tool  in  dealing  with  a  number  of  stochastic 
control  problems.  To  make  local  time  a  bit  less  esoteric,  we 
have  undertaken  to  write  a  book  on  its  general  nature  motivated 
by  its  applications  to  control.  This  book  will  present  many  of 
the  problems  discussed  here,  and,  so  as  to  be  nearly 
self-contained,  it  will  also  include  a  full  treatment  of  Brownian 
motion,  stochastic  integration  with  respect  to  continuous, 
squar e - i n t eg r ab 1 e  martingales,  and  strong  and  weak  solutions  of 
stochastic  differential  equations.  It  is  our  goal  to  make  the 
topics  treated  in  this  proposal  accessible  to  the  matheamtics. 


engineering  and  economics  communities. 

Upon  the  expiration  of  this  grant,  the  book  was  about  half 
completed.  A  table  of  contents  for  the  entire  book  is  provided 
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Research  on:  Optimal  Control  with  Diminishing  or  Zero  Cost  of  Control. 


In  studying  the  effect,  in  deterministic  control  problems,  of 
very  marked  dependence  on  system  state,  of  the  asymptotic  growth 
of  cost  of  control,  we  are  preparing  the  ground  for  application  of 
control  to  relatively  sophisticated  systems.  In  the  long  run  we 
hope  to  analyze  control  properties  for  multidimensional  nonlinear 
systems  such  as  those  occurring  in  continuous  elastic  structures 
stressed  to  configurations  near  their  yield  points.  It  is  well 
known  that  such  systems  are  subject  to  "fracture",  and  one  outgrowth 
of  the  present  work  may  be  an  understanding  of  whether  fracture  can 
arise  in  consequence  of  the  minimizer  for  the  energy  integral,  although 
continuous,  having  a  gradient  which  is  infinite  along  a  low  dimensional 
portion  of  the  given  structure.  In  the  near  term  we  expect  to  further 
clarify  the  one  dimensional  case  through  the  use  of  classical  analysis 
on  spaces  of  absolutely  continuous  functions. 

Our  work  with  Professor  John  Ball  seems  to  have  been  the  first 
to  demonstrate  regular  problems  with  such  anomalies  as  failure  of 
the  minimizer  to  satisfy  the  integrated  Euler-Lagrange  and  "energy" 
equations  (i.e.,  the  maximum  principle),  as  well  as  the  existence 
of  a  non  zero  gap  between  the  actual  minimum  value  of  the  cost 
integral  and  the  lower  limit  it  assumes  for  continuous  controls 
("Lavrentiev  phenomenon").  Other  researchers  are  currently  becoming 
involved  in  related  issues  involving  the  limitations  of  classical 
control  and  variational  theories.  The  Lavrentiev  phenomenon  has 
important  implications  for  the  numerical  solution  of  control  and 
variational  problems  since  numerical  approximation  inevitably 
involves  smooth  admissible  functions.  An  investigation  is  under 
way  to  analyze  the  behavior  of  corresponding  diffusion  type  stochastic 
control  problems  (in  which  the  cost  of  additional  control  has  similar 
asymptotic  growth  properties) ,  in  order  to  test  whether  the  limit 
of  small  noise  perturbation  will  provide  more  easily  treated 
approximation  methods  for  the  deterministic  problem. 

The  long  range  purpose  of  our  work  in  stochastic  control  is 
to  expand  the  set  of  models  which  are  well  understood  analytically. 
Except  for  the  linear-quadratic-Gaussian  regulator  and  the  models 
presented  by  Benes,  Shepp  and  Witsenhausen  in  1980,  there  are  very 
few  stochastic  control  problems  which  can  be  "solved"  in  any  sense 
other  than  numerically.  We  have  attacked  specific  problems  in 
"bang-bang"  and  singular  stochastic  control;  the  former  have  arisen 
in  guidance  and  the  latter  in  inventory  and  queue  control.  Our 
work  in  bang-bang  control  uses  the  tools  of  Brownian  local  time 
and  path  decomposition,  both  of  which  are  now  under  active  investi¬ 
gation  by  probabilists .  The  singular  control  problems  date  back 
to  work  by  Chernoff  in  1968,  but  are  only  now  receiving  widespread 
attention. 

In  the  next  year  we  expect  to  learn  more  about  local  times 
and  path  decompositions,  and  we  also  expect  to  solve  other  specific 
control  problems  using  these  techniques.  In  contrast  to  singular 
control,  which  now  benefits  from  much  activity,  we  are  aware  of  no 
other  current  work  in  the  important  and  interesting  area  of  bang-bang 
control,  so  we  expect  our  emphasis  to  switch  somewhat  in  that  direction. 


Recent  Developments  in  Deterministic  and 
Stochastic  Optimal  Control  (AFOSR-82-0259) 

V.  J.  Mizel  and  S.  E.  Shreve 
February  8,  1984 

During  the  past  year,  our  investigation  of  the  influence,  in 
deterministic  control  problems,  of  varying  the  growth  rate  for 
cost  of  control  has  led  to  two  significant  developments.  First, 

t 

it  has  been  demonstrated  that  even  for  regular  one-dimensional 
Lagrange  type  control  problems  {with  polynomial  integrand) ,  under 
growth  conditions  which  ensure  existence  of  a  global  absolutely 
continuous  minimizer,  one  can  have  a  global  minimizer  which  fails 
totally  to  satisfy  the  integrated  form  of  the  maximum  principle! 
More  specifically,  examples  have  been  constructed  of  calculus  of 
variations  problems  in  one  real  function  whose  global  minimizer 
satisfies  neither  the  integrated  form  of  the  Euler-Lagrange 
equations  nor  the  integrated  form  of  the  DuBois-Reymond  ("energy") 
equation.  These  examples  are  apparently  the  first  known  uniformly 
regular  problems  possessing  absolutely  continuous  but  non-Lipschitz 
minimizers.  The  feature  of  the  integrand  which  permits  such  an 
anomaly  is  that,  despite  a  uniformly  quadratic  lower  growth  in 
the  cost  of  control,  the  polynomial  growth  of  cost  of  control  is 
affected  in  an  essential  way  by  the  choice  of  state  at  which 
control  is  applied. 

Second,  it  was  found  that  the  long  forgotten  "Lavrentiev 
phenomenon"  (discussed  in  the  recent  book  of  Cesari)  can  readily 
occur  in  such  problems.  That  is,  one  can  have  a  problem  with  a 


rion-Lipschitz  minimizer  such  that  Lipschitz  functions  cannot 
simultaneously  approximate  the  minimizer  itself  and  the  cost 
incurred  with  this  minimizer;  the  infimum  of  the  costs  attained 
by  Lipschitz  trajectories  can  be  strictly  greater  than  the  cost 
achieved  by  the  absolutely  continuous  minimizer!  The  implications 
of  this  phenomenon  for  the  numerical  study  of  control  problems  are 
profound — numerically  generated  costs  are  inevitably  connected  with 
Lipschitz  trajectories.  Hence  the  numerical  study  of  such  control 
problems  will  invariably  lead  to  overly  pessimistic  conclusions 
concerning  their  optimal  cost  unless  techniques  are  consciously 
followed  to  bypass  the  Lavrentiev  phenomenon. 

The  objective  of  our  work  in  stochastic  control  has  been 
to  study  problems  in  which  Control  is  exercised  in  a  "bang-bang" 
fashion,  (i.e.,  it  switches  discontinuously  as  -the  state  process 
crosses  a  "switching  surface") ,  or  in  which  control  is  singular, 
(i.e.,  it  can  be  thought  of  as  the  limit  of  impulse  controls  as  the 
sizes  of  the  impulse  approaches  gd  while  the  duration  approaches 
zero) .  The  former  problems  first  arose  in  the  context  of  rocket 
guidance  with  the  desired  trajectory  serving  as  a  switching 
surface;  the  latter  problems  appear  in  inventory  control  where 
demand  and  supply  processes  provide  impulses  to  the  inventory 
process.  We  have  thus  far  succeeded  in  computing  the  statistics 
of  bang-bang  controlled  processes  in  some  interesting  one-dimensional 
cases,  and  we  believe  the  collateral  discoveries  we  have  made  about 
Brownian  local  time  and  path  decompositions  are  of  independent 
interest.  We  have  also  shown  how  to  reduce  some  fairly  large 
classes  of  singular  control  problems  to  much  studied  problems 
in  optimal  stopping.  We  expect  this  reduction  to  result  in 
interplay  between  the  two  kinds  of  problems  so  as  to  enhance 
understanding  of  both. 


